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ABSTRACT 



A description is given of a program designed to improve 
technology transfer and utilization in a classical organization 
consisting of a research and development activity and several 
engineering oriented user activities. The effectiveness of the 
technology transfer and utilization program is longitudinally 
studied. Three year comparisons of several parameters are 
reported. A benefit evaluation decision model is introduced in 
order to improve the accuracy of the cost/benefit evaluation of 
the technology transfer and utilization program. Using this model 
further longitudinal comparisons are made. Finally the general 
usefulness of the benefit evaluation decision model is argued by 
showing that for each dollar of investment this particular program 
gave a present value return of $2.72 in benefit. Several addi- 
tional comparisons that could be made are suggested. 



Technology Transfer, Introduction 



The concept of "technology transfer" is difficult to define 
since its meaning seems to vary with the audience discussing it. 

In general, however, the transfer of technology differs from the 
usual dissemination of scientific knowledge in that it is more 
concerned with the usage of technological information obtained 
through research/development effort. "Any mechanism developed 
for transferring technology from its origin to its usage should 
be directed more toward suggesting methods and areas of applica- 
tion than toward merely publishing scientific results in technical 
documents to be filed in technical sections of depositories" 
(Doctors, 1969, p. 56) . 

In the United States the federal laboratory system repre- 
sents a vast resource of science and technology, with over 469 
major research and development (R&D) installations (RCSG, 1973, 
p. iii) . These agencies have been producing technological reports 
at about 50,000 per year (Olken, 1972, p. 9). Over the past 
decade the executive branch of the government has emphasized the 
need for the federal government to actively disseminate this 
technology to the public and private sectors in an effort to 
increase the economic benefit of the information. 

At present the federal government is supporting a number 
of technology transfer programs in several agencies and depart- 
ments; however, it is very difficult to measure their results. 

"... there is a need for comprehensive experiments ... which would 
investigate the acquisition, evaluation, and dissemination of 



technical information and the measurement of its use after 
dissemination" (Doctors, 1969, pp. 7-8). 

Objective of the Study 

The objective of this study was to use longitudinal data 
to evaluate the effectiveness of a technology transfer and utili- 
zation program that had been funded in 1971 by the Naval 

1 . . 

Facilities Engineering Command. Specifically the program under 

study was the Facilities Engineering Support Office (FESO) of the 

2 

Navy's Civil Engineering Laboratory (CEL). 

In the process of upgrading the evaluation techniques a 
benefit evaluation decision model was introduced. 

A second objective of the study was to show that the benefit 
evaluation decision model is a useful tool in terms of providing 
a meaningful method of quantifying the benefits of a technology 
transfer and utilization program. 

General Background 

The Naval Facilities Engineering Command (NAVFAC) executes 
a program of research, development, test and evaluation (RDT&E) 
for shore facilities, advance base and amphibious operations, sea 
floor structures, environmental control and those aspects of 



This research was supported in part by the Naval Facilities 
Engineering Command, Washington, D. C. The principle researchers 
on the project were J. A. Jolly, J. W. Creighton, J. E. Hendrickson 
and W. G. Fisher, Jr. 

2 

The assistance rendered by Mr. Eugene H. Early, Head of the 
Facilities Engineering Support Office of the Civil Engineering 
Laboratory, was greatly appreciated by the researchers and is 
hereby gratefully acknowledged. 



2 



weapon systems related to its mission. A significant portion of 
the emphasis of NAVFAC's program is to provide RDT&E which will 
benefit the Navy's shore facilities in efficiently and effectively 
meeting their independent missions. NAVFAC's link to the shore 
facilities is primarily through the Engineering Field Divisions 
(EFD's), Public Works Centers (PWC's), Public Works Departments 
(PWD's), and its construction program with Officers in Charge of 
Construction (OICC's) and Resident Officers in Charge of Construc- 
tion (ROICC's). Figure 1 shows these relationships. A major por- 
tion of NAVFAC's RDT&E effort is assigned to CEL in the form of 
specific research projects. 

The Civil Engineering Laboratory (CEL) is the principle 
research, development, test and evaluation center for shore and 
sea floor facilities and for support of Navy and Marine Corps 
construction forces (NCEL Inst. 1970, 1972). The staff of CEL 
consists of approximately 320 personnel, 150 of whom are profes- 
sional engineers. 

Funding for CEL's FY'74 program exceeded $13 million. For 
FY'74 the bulk of CEL's efforts, about 76%, were in exploratory 
development (applied research) . Roughly 1% of the effort was in 
research, and the balance, about 23%, was in advanced engineering 
and operational systems development (RAP BRIEF 1974, p. ii) . 

NAVFAC's Technology Transfer Program 

The Navy's RDT&E funds are administered by a division of 
NAVFAC which has the responsibility of insuring that the input and 
output of R&D information is transferred between all levels of the 
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Figure 1. Navy's Civil Engineering Activity 



An qrg^nizational diagram showing the flow of 
technical information between the Naval Facilities 
Command Headquarters, the Civil Engineering Labora- 
tory qnd the various engineering field activities. 



Navy to insure that the maximum benefit from R&D expenditures is 
obtained. Two specific organizational innovations are funded by 
NAVFAC in an effort to better coordinate the flow of R&D informa- 
tion from and to the field operating units (see Figure 2) : 

1. The Facilities Engineering Support Office (FESO) . 

2. The Engineering Field Division Liaison Billets. 

The Facilities Engineering Support Office 

The FESO organization was established by CEL and funded by 
NAVFAC to perform the function of coordinating ser'/ices and com- 
munications related to RDT&E assistance to Naval shore activities. 
The specific objective of this program was to provide RDT&E assist- 
ance to Naval shore activities by having CEL perform short-term 
ser^/ices to determine the relative value and suitability of new 
materials, equipments, processes and construction or maintenance 
procedures (NCEL Inst. 1971). 

Conceptually, the FESO is in a liaison position and as such 
consists at the present time of one civilian. He administers the 
functions of the office by coordinating and recording the flow 
of information between the field units and the specific laboratory 
individuals having expertise in the area of the inquiry. In addi- 
tion, this office is tasked with assuring that field units are 
knowledgeable of the lab's current programs and the availability 
of the FESO service. This advertisement function is pursued 
through various media including leaflets, bulletins and site visits. 

24 hour phone service is also maintained by FESO to handle and 
record incoming calls world-wide. In general, CSL's internal 
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Figure 2. NAVFAC Technology Transfer Program 



The organizational action taken by CEL 
was to establish a Facilities Engineering Support 
Office at the Civil Engineering Laboratory. NAVI AC 
established a Liaison Representative at each of the 
Engineering Field Divisions. 



guidelines for expenditures of the funds allotted to the FESO 
provides that assistance requests involving twenty man-hours or 
less are approved at Division/Department level. Requests for ser- 
vices involving more than twenty man-hours (or $500) or frequently 
recurring requests of a similar nature are categorized as job 
order requests. These requests must be approved by the technical 
director . 

The Engineering Field Division (EFD) Liaison Billets 

The RDT&E liaison billet was intended to facilitate communi- 
cation between field activities and NAVFAC in the area of RDT&E 
and to serve to improve and expedite the evaluation of new ideas, 
concepts, procedures, new materials, equipment, and to insure feed- 
back to NAVFAC for further dissemination Navy -wide. They study 
technical problems unique to the field forces and transmit to NAVFAC 
proposals for research effort to initiate corrective measures (NAVFAC 
3900.6 of 13 July 1966). They also serve as the CEL FESO primary 
point of contact at each of the EFD's. 

Evaluation of FESO Benefits ^ 

1. Quantity of Requests 

The number of requests recorded by the FESO office were as 
follows: FY1972 = 281, FY1973 = 349 and FY1974 = 396. Under 

the assumption that an increasing number of requests indicated 
increasing benefit to the field activities, the program has been 
expanding well. 

^Survey techniques and details of each year's evaluation may be 
found in the technical reports listed in the reference [Jolly and 
Creighton, 1973, Jolly et al. Sept. 1974, Jolley et al . Dec. 1974]. 
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2. Categories of Project Requests 

Identifying specific categories of requests by project 
type is beneficial for determining major problem areas encoun- 
tered by the EFDs. This information could be utilized in empha- 
sizing future R&D efforts as well as affecting manpower require- 
ments within the laboratory. By far the most common request in 
each year studied was in the area of paint and coatings. These 
accounted for between 18% and 25% of the total- Other problem 
areas achieving some significance, and ranging from 3% to 12%, 
included water pollution, classified disposal, structural, corro- 
sion, concrete, electronic, pavement, mechanical, electrical, an4 
miscellaneous pollution. 

3. Means of Communication 

The communication link to and from CEL was considered of 
vital importance to the program. The communication system must 
be accessible to those seeking information and also have the 
ability to transmit comprehensive and timely data. Survey data 
showed that the telephone was the predominant means of communicat- 
ing requests from the Field to CEL. The use of the telephone 
increased from 60.2% of the total request in FY'72 to 75.5% in 
FY'74. Contributing factors to this sizable increase were the 
establishment of a 24 hour answering service in the FESO office 
and emphasis in CEL advertisement of their accessibility by tele- 
phone. The survey information also indicated that the telephone 
was a major means of communicating responses from CEL to the field, 
41.5% in FY'72 and 54.8% in FY'74. It should be noted, however. 
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that the data used to calculate the above percentages was based 
on initial request and initial response. In many cases, letter 
and message follow up documentation of both request and responses 
were used. 

4. Response Time 

One of the objectives of the program was to provide a rapid 
response to field requests. Results of measuring this parameter 
are shown as Figure 3. It is interesting to note that approxi- 
mately one-half of all requests were answered within seven days 
of the initial request. 

5. Utilization of Assistance 

An important measurement of the program effectiveness is 
the determination of the degree to which the information obtained 
from CEL was used by the requesting activities. Figure 4 is a 
comparison of the productive versus the unproductive contacts. 

The results are expressed in percentage. It is difficult to 
attach any statistical significance to the relative small change 
that was observed over the three year span. 

6. Dollar Benefit of Assistance 

The measurement of the dollar benefit of the NAVFAC tech- 
nology transfer program involving the, CEL and the EFD's was con- 
sidered desirable. The measures discussed up to this point in 
this paper tend to indicate that the program was working well, 
but they are not expressed in dollar benefits. In fact, each 
years evaluation questionnaire attempted to determine dollar 
benefits by asking the requestor to quantify within specific 
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Period 


1 day 


2 days 


7 days 


14 days 


30 days 


FY '72 


26 


34 


45 


57 


76 


FY’ 73 


31 


35 


52 


65 


76 


FY' 74 


27 


37 


57 


63 


75 



Figure 3. Percent of Total Requests 
Answered Within Time Period 



Shown is the cumulative percent of requests that were 
answered within a specific time period. As can be seen 
approximately one-half of all requests wer($ answered 
within seven days. 





FY1972 


FY1973 


FY1974 


Productive Output 


72.8 


82.8 


79.0 


Unproductive Output 


27.2 


17.2 


21.0 



Figure 4. Percentage Utilization of the Output 
Resulting from Requests 



It is quite apparent that the year to year change in 
productive versus unproductive output from the requests 
is such that it is difficult to attach any statistical 
significance to the small change. It would appear that 
a 70 to 80 percent utilization over time can be expected. 
The results shown are from survey data. 



ranges the estimated five year operating cost of individual 
projects, with and without the assistance from CEL. The differ- 
ence between the with and without assistance cost was used as the 
benefit. Using this method the results reported are given in 
Figure 5 . 

In compiling the data reported in Figure 5 it was observed 
that only a small fraction of the requestors that completed the 
questionnaire were willing to assign a dollar value to their use 
of the CEL information. For example in FY1972 of the sample of 
83, only 12 or 14.5% gave dollar values such that dollar benefit 
could be calculated. It was somewhat better in FY1973, the sample 
was 93 and 28 or 30% gave dollar values such that dollar benefit 
could be calculated. Even so, it was felt that the true value of 
the benefit was not being evaluated using this system. The next 
few paragraphs will describe in detail an alternate method of 
obtaining a dollar value for the benefits derived from the trans- 
fer of technology by the CEL to the EFD's. 

Benefit Evaluation, The Problem 

Objectively quantifying the benefits of the CEL technology 
transfer effort, as indicated in the previous paragraphs, was at 
best highly subjective and lead to varying results. There existed 
a considerable range of latitude when an attempt was made to 
quantify the dollar value of the benefit derived from a piece of 
information. 

A specific recommendation to solve a particular problem 
may easily be quantified if it will reduce out of pocket expenditures 
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to achieve identical results. Factors such as quality of output 
could, however, tend to cloud even this type of calculation. 
Quantifying benefits derived from one piece of information which 
is only a part of the total information required to arrive at a 
decision leaves room for even greater subjectivity. At the other 
end of the scale, quantifying an intangible benefit such as in- 
creased moral, safety, and general information probably is the 
most subjective measurement of all (Quinn, 1959, p. 11). 

In essence, any attempt to quantify the benefit of informa- 
tion is necessarily highly subjective, and recognition of this 
fact is an underlying consideration in the development of a new 
approach which is presented here. 

Benefit Evaluation, The Model 

It was determined that the major issue in evaluating the 
benefit from a technical recommendation was the categorization 
process. A system was needed to test whether or not a benefit 
resulted and if a benefit resulted then to what extent the recom- 
mendation was responsible for the final benefit. This categoriza- 
tion may be accomplished by the benefit evaluation decision model 
shown as Figure 6 (Hendrickson and Fisher, 1974, p. 33-47). 

Using Figure 6 and starting with an answer to a request for 
technical information the question at A is, "Did the answer to 
the request for technical information result in a benefit either 
tangible or intangible?" If the answer to the question is no, 
then the analysis terminates and that information request or recom 
mendation is counted as having zero benefit. If the answer is yes 
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Year 


Total 

Requests 

Number 


Sample 

Size 


Estimate 

from 

Sample 


Extrapolated 
to Total 
Requests 


FY' 72 


281 


29.5% 


$28,000 


$ 94,915 


FY' 73 


349 


26.6% 


$46,000 


$172,932 


FY' 74 


396 


26.5% 


$77,000 


$290,566 



Figure 5. Benefit from CEL Assistance Program 



Estimate of benefits using survey questionnaire data. 
This does not include extraordinary benefits reported, 
i.e. FY' 73 = $15,000 and FY ' 74 = $187 , 000 . 




Figure 6. Benefit Evaluation Decision Model 

Start at decision A, a series of decisions are shown that make it 
possible to evaluate the dollar benefit of the answer to a technical 
question supplied to an engineering organization by a research 
laboratory. 





then the decision next in order on the decision model is B, "Is 
the information general so as to have no immediate value or is 
the information specific and useful?" If the information is 
general in nature it may be filed for use at some future date 
and the present value of the dollar benefit is assumed to be the 
acquisition cost. In contrast if the information is specific, 
then the path leads to decision C. At decision C the informa- 
tion must fall into one of two classifications, either partial 
information for a decision or complete information for a decision. 
When the information is partial, that is, when the information is 
only part of the total information used to arrive at a decision, 
then it is appropriate and necessary to determine the percent of 
contribution to the total decision. This process is shown as 
block D of the model. When the information is substantially 
complete and self contained in terms of influencing the decision 
then one hundred percent credit for the benefit is allocated to 
that information request. Block E shows this assumption. 

Both of these information evaluation channels then lead to 
an action decision block F. At decision point F there are 
four alternatives. If the project was implemented then the deci- 
sion route is to G where it is necessary to determine if the 
results were estimable. In many cases a specific dollar benefit 
can be assigned at this point of the analysis. 

A project may be delayed, but with a specific plan for 
future implementation. This decision is represented by H. Some 
future action benefits can be estimated in terms of a dollar 
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return, others are more intangible. This decision is made at 
point H. 

The third branch of F is for those projects that have a 
delayed plan because testing is required. The estimate of bene- 
fit is based on the assumption that if the tests are successful, 
then the project will be implemented. If the project is imple- 
mented there should be a resulting benefit. For some projects 
the potential benefit, once the project is implemented can be 
estimated (decision point J) but for other projects the benefit 
may be only intangible such that it would be mpst difficult or 
perhaps impossible to assign a specific dollar value to the benefit. 

The fourth and last choice, at action decision block F, 
is delayed-plan-study-required . This is similar to the delayed- 
plan- test- required except that a feasibility or economic study 
may be necessary in order to determine the worth of the project. 

As part of the study additional tests may be necessary and/or 
desirable in order to reduce the risk associated with implementing 
the recommendation. This branch has the lowest probability of 
resulting in a dollar benefit, because of the several uncertain- 
ties that must be considered. If it should be implemented, it, 
like the other decisions of decision block F, must proceed to 
the next level of decision (decision block k) in order to deter- 
mine if the benefit is estimable and therefore a specific dollar 
benefit pr if the benefit is intangible and therefore difficult 
to evaluate the benefit. 
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Perhaps the most important function of the benefit 
evaluation model, is to provide an organized system of classifi- 
cation or categorization of the benefits to answers to requests 
for technical information. When the categorization process is 
completed, then it is possible to continue toward the objective 
of determing the benefit derived from the recommendation. This 
requires the determination of the percentage of contribution of 
the recommendation to the final product decision, the assignment 
of probabilities to the likelihood that the project will be imple- 
mented and, a determination of the dollar base for each benefit. 
These issues will be discussed in the next few paragraphs. 

Determining the Coefficients and Dollar Base 

To use the benefit evaluation decision model it is necessary 
to (1) assign values to the factor, information contribution per- 
centage (decision D) for each benefit that resulted from the use 
of technical information provided by CEL to (2) assign proba- 
bilities to the likelihood of implementation of expected actions 
(decision F) , and to (3) determine the dollar base for each bene- 
fit that resulted from the use of technical information provided 
by CEL. A discussion of each follows: 

1. Factor, Information Contribution Percentage 

The coefficient for the factor, information contribution 
percentage, was assigned to each case on an individual basis. 

The total information available in order to make a decision on 
implementing a project was reviewed by the researchers. The fac- 
tor coefficient was selected from the range of 0.01 to 1.0 based 
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on the relative effects that the CiiL provided information had on 
the selection of the most beneficial alternative available. Other 
considerations in the selection of a factor coefficient were the 
availability of the information from other sources and the rela- 
tive benefit of the next best alternative that may have been 
selected had CEL provided information not been available. 

2. Implementation Probabilities 

Responses to requests for technical information which were 
considered beneficial (Decision A) were classified by the benefit 
decision model into one of the following categories which are 
listed in descending order according to the probability of imple- 
mentation of the information provided; 



Final Decision 



Decision Model 
Path 



Probability of 
Implementation 

value range 



1. Information has been 
implemented 



A, beneficial 

B, specific info. 

C, evaluate 
E, action 

G, implement 



1.0 



2. Specific plan to 
implement 



A, 

B, 

C, 
F, 



H, planned for future 0.5 0.4-0. 6 



3. Implement if tests are 
successful 



A, 

B, 

C, 
F, 



J, delay plan (test) 0.3 0.2-0. 4 
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Implement if study indicates 
benefits are probable 



value 



range 



A, 

B, 

C, 

F, 

K, delay plan (study) 0.2 0.1-0. 3 

General information not 
specific to a project A, 

B, file for future use 

The probabilities shown were based on the experience of the 
researchers. It would be appropriate to adjust these probabilities 
depending upon the history of the organization and/or the experi- 
ence of the researcher using the benefit evaluation decision model. 

3. Dollar Base for Model Calculations 

Projects with dollar savings specifically identified on 
the FY'74 questionnaire were classified as estimable. If the 
identified savings were of the one-time type (versus recurring) , 
the amount so identified was used as the project benefit dollar 
base. The benefit credited to CEL assistance in such a case was 
the project benefit dollar base reduced by the factor for informa- 
tion contribution and the factor for implementation probability. 

If estimable identified savings were of the recurring type, the 
project benefit used was the present value of the first five years 
of savings. A present value factor of 3.935 for a steady cash flow 
throughout the year utilizing a 10% rate of return was used as the 
project benefit dollar base. Again, the benefit credited to CEL 
assistance was the project benefit dollar base reduced by the 
factor for information contribution and the factor for implementa- 
tion probability. 
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Beneficial projects resulting from CEL supplied 
recommendations, which did not have specifically identified 
dollar savings, generally fell into areas where the benefit was 
in the fopm of improved operations, better morale, increased 
safety, improved quality, etc. In FY'72 and FY'73 surveys, 
benefits of projects of this type were left unquantified. With 
the exception of responses to requests which fell into the cate- 
gory, general information, each response to a request should have 
had an identifiable benefit even though it was not readily 
quantifiable in terms of direct dollar savings. Each response to 
a request for technical information could in some way be identi- 
fied with an implemented or proposed project, the magnitude of 
which was normally relatively easily quantified. 

The assumption accepted was that in order to commit funds 
to a project a decision maker must, whether he realizes it or not, 
expect a return in future benefit which is some percentage greater 
in present value than the initial outlay. This percentage may 
vary from decision maker to decision maker and will even vary with 
time under varying circumstances for any decision maker. Even 
though the investment return would be expected to vary, it is 
assumed that the decision maker (in any organization) , who decides 
on implementation of a project based on supplied technical infor- 
mation, would be of a quality such that the results of their 
decision over the long run and on the average would yield a posi- 
tive benefit. In the case of evaluating the output of CEL assistance 
approximately 100 of 295 requests for information in the FY'74 sur- 
vey fell into this unestimable category. The benefit then, for the 
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projects in this category, was the project investment reduced 
by the factor for information contribution percentage and the 
factor for implementation probability. 

Application of the Model to FY'74 Data 

The FY'74 survey included 295 requests. FY'74 question- 
naires were completed on each of these requests, 105 through the 
six EFD RDT&E representatives, and the remainder by researchers 
at the Naval Postgraduate School, Monterey, California, based 
on telephone interviews with the original requestor. Of these 
295 questionnaires, 233 indicated that the requestor considered 
that he had received beneficial information. The remaining 62 
questionnaires indicated that the request yielded no beneficial 
information. However, among these were 40 cases which indicated 
that there were extraordinary circumstances indicating that these 
cases should not be included as zero benefit requests, but rather 
should be eliminated from the sample for purposes of cost benefit 
analysis and study. Most common among these extraordinary cases 
were : 

1. The request was merely a followup on a previous 
request. The benefits were totally included on the 
original request number and were listed on this one 
as zero to avoid double counting. 

2. The lab had requested additional information on 
the problem from the originator but the information 
was not provided. 

Table I shows the tabulation of the numbers of cases and 
the total benefits calculated for each code based on the code 
numbers assigned by Figure 7. This table shows three figures 
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Benefit 

Code 



Probability of 
Implementation 



Benefit Estimate 



01 

02 

03 

04 

05 

06 

07 

08 
09 



Information has been implemented 

Same 

Specific plan to implement 

Same 

Implement if tests are successful 

Same 

Implement if study indicates benefits 

Same 

General information not specified to 
a project 



Estimable 

Not estimable 

Estimable 

Not estimable 

Estimable 

Not estimable 

Estimable 

Not estimable 

Acquisition cost 



Figure 7. Benefit Codes According to Likelihood 
of Deriving Benefits 



Code numbers are assigned to each benefit path 
of the model. These code numbers are used to 
identify the decision combination when specific 
dollar benefits are discussed. 



for the benefits in each category. The three figures represent 
calculations based on use of the mean, the high, and the low values 
of the probability of implementation. Figure 8 shows the plotted 
values indicating the increasing uncertainty as the cumulative 
benefits progress to include more subjective estimates. 

A Cumulative Benefit Curve 

The curve. Figure 8, resulted from drawing smooth curves 
through the points derived from Table I. They represent graphi- 
cally the fact that, as the benefits of a greater percentage of 
cases in the sample are quantified (a greater number of subjec- 
tive estimates are included) , the total estimate of cumulative 
benefits becomes more subjective. The vertical distance between 
the "high" and the "low" curves at any point on the horizontal 
scale represents the range within which the estimate could reason- 
ably be expected to vary due to differing personal values of 
estimators or decision makers. 

Although not by any means an analytical proof, the curve 
tends to intuitively verify the applicability of the model. As 
intuitively expected, the benefits from the highly intangible 
cases are less than those from the more tangible ones, as indi- 
cated by the decreasing slope of the curves. This intuitive 
approach is further strengthened by the observation that a deci- 
sion maker will generally give less weight to the more intangible 
benefits when confronted with the choice of whether or not to 
spend today's very tangible dollars for future benefits. 
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THOUSANDS OF DOLLARS 




NUMBER OF REQUESTS 
BENEFIT CODE 



Figure 8. CEL Assistance Dollar Benefits vs Number of 
Requests with Requests Ordered According to 
Probability of Estimate 



This figure shows the curves of quantified benefits 
utilizing high, mean and low probability of imple- 
mentation. The slope decreases and the range of 
accuracy widens as greater numbers of requests in the 
sample are quantified. 



Benefit 


Number of 




Value 




Code 


Requests 


Low 


Mean 


High 


Not counted 


40 


— 


— 


— 


Zero benefit 


22 


0 


0 


0 


01 


27 


$254,361 


$254,361 


$254,361 


02 


51 


9,260 


18,525 


37,051 


03 


6 


15,399 


19,250 


23,099 


04 


38 


11,816 


29,540 


70,896 


05 


8 


41,690 


62,534 


83,379 


06 


8 


3,485 


10,455 


27,880 


07 


1 


236 


472 


708 


08 


12 


532 


2,130 


5,890 


09 


82 


unquantified 






295 


$336,779 


$397,267 


$503,264 



Table I. Quantified Benefits for FY'74 PESO Operation 



Dollar benefits are shown according to benefit code 
(see Figure 7). The dollar benefits are calculated 
using the Benefit Evaluation Model. 



For the purposes of the analysis made throughout the 
remainder of this study, the benefit value utilized will include 
Benefit Codes 01 through 08 only and will be based on mean factor 
values . 

A Comparison with Past Years ' Surveys 
^ 

The FY'74 survey methods, in large part, have resulted 
from experience gained by FY'72 and FY'73 surveys. Many questions 
have been changed to some degree each year in an effort to obtain 
better data. One disadvantage of the continuous change is the 
fact that the figures are not strictly comparable from year to 
year. This comparison is necessary, however, in order to show 
trends over time. 

Data from the FY'74 survey is similar to that of the earlier 
surveys in all areas except the dollar value estimates of benefits. 
The major differences between the FY'74 and earlier data in the 
area of quantifiable benefits is shown below: 

FY'72 and FY'73 Surveys 

1. Only readily estimable cases which had been implemented 
were included. 

2. Of the total Shore Facilities requests received 40% were 
surveyed. 

3. Where annual savings were identified, the project benefit 
was calculated as five times the annual savings. 

4. The total project benefit was used as the benefit of CEL 
assistance . 

FY'74 Survey 

1. Any benefit which could be reasonably quantified was used. 
Projects with future implementation potential were included 
and the benefits thereof were reduced by applying a proba- 
bility factor. 
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2. All Shore Facilities requests were surveyed. 

3. Where annual savings were identified, the present value 
of five-year savings was used. 

4. An information-% - factor was applied to the project 
benefit to arrive at the benefit contribution. 

The dollar benefit figures computed by FY'74 survey methods must 
be adjusted to be comparable to the earlier survey figures for 
purposes of examining trends. 

The cases in the FY'74 survey which were classified into 
Benefit Code 01 are only partly equivalent to the cases quantified 
in the FY'72/FY'73 surveys. The quantified benefits applicable 
from this code totaled $66,662.^ Adjusting this figure downward 
by a factor of 0.4 to $26,665 compensates for the 40% sample of 
the FY'72/FY'73 surveys. Further adjusting upward by dividing by 
.37 to $72,068 compensates for use of the information-% factor in 
FY'74. Approximately 83% of the readily estimable savings in the 
FY'74 survey were of the annual recurring type. The resulting 
adjustment for use of the present value factor (472,068 x 3.9 x 
5 X 0.83) yields $76,688. 

By a similar method, the FY'72 and FY'73 survey data can 
be adjusted to reflect results as if the FY'74 method was applied 
to all three years. Figure 9 shows a graphical comparison of all 
three years utilizing both the FY'72/FY'73 and the FY'74 survey 
methods. The extraordinary cases have been eliminated since it 
is inappropriate to include them in projections either backward 

^This figure excludes one project, quantified at $187,000. This 
project is considered extraordinary and not of a recurring type 
and should, therefore, not be included at this point. 
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Thousands of Dollars 



210 




Figure 9. CEL Assistance Benefit Trends (Thousands 
of Dollars) 



The bar graph shows the CEL assistance benefit using 
(1) estimates of the requestors and (2) using esti- 
mates from the Benefit Evaluation Decision Model. 



or forward. It should be noted, however, that over the long term 
extraordinary cases of this type may constitute a significant 
portion of relaized benefits. 

Benefit Analysis Using Selected Topics 

This study evaluated only 295 requests of the total 396 
that the PESO office recorded during FY'74. With respect to the 
total requests the analysis represented 75% of the CEL assistance 
effort. The value of the benefits attributable to these 295 re- 
quests, as measured by the model amounted to $395,269 (see Table 1). 
Of the total cost of the CEL assistance for FY'74 only the portion 
of the total cost applicable to the 295 requests would be appropriate 
for a cost/benefit analysis. These costs are: 

273 short-term requests $48,841 

22 job order requests $66,346 

FESO administrative expenses, proportioned 
as to total assistance requests processed 

295/396 X 41,494 = 30,911 $30,911 

$146,098 

Evaluation of the CEL assistance program within the frame- 
work developed above indicates that there was a present value return 
of $2.72 for every dollar spent ($397, 267/$146 , 093) . 

The primary purpose of the CEL assistance, as discussed pre- 
viously was to provide a rapid response to requests for assistance 
from field operating units. The rapid response requests were re- 
corded separately. If these short term rapid response requests only 
were considered, then the appropriate costs should include direct 
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expenditures for the 273 short-term requests, plus an appropriate 
proportion of the administrative costs. Expenditures for these 
short-term requests totaled $77,447 (i.e. $48,841 = [273/396 x 
$41,494]). The dollar value of the benefits from these 273 requests 
was $184,444. The return of the CEL assistance program for these 
short-term requests was $2.38 for each dollar spent. 

When the dollar benefit of each project can be determined 
it is then possible to construct charts for analysis purposes. For 
the CEL assistance these data were used to construct the following: 

1. Comparison of costs and benefits with respect to project 
type, i.e. paints, pavement, pollution corrosion, etc. 

2. Comparison of cost and benefits by benefit code, i.e. 
implemented, specific plan for future implementation, etc. 

3. Comparison of cost and benefit by user group, i.e. EFD's, 
PWC's, OICC's, etc. 

4. Comparison of cost and benefit by originator, i.e. civilian 
engineer, military engineer, technician, scientist, etc. 

Other comparisons meaningful to a specific organization or 
evaluation problem should be obvious depending on the needs and/or 
objectives of the study. 

Summary and Conclusions 

This longitudinal study of a classical organization with a 
specific structure to promote technology transfer and utilization 
has carried the cost/benefit analysis beyond the usual comparison 
of number of requests, response time, and estimate of tangible 
benefits. A benefit evaluation decision model was introduced 
that provided a means for categorizing technical information and/or 
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technical recommendations. The model considers both tangible 
and intangible benefits. After the technical information and/or 
technical recommendation is categorized and a dollar benefit base 
assigned, the dollar benefit is adjusted (1) according to the per- 
cent of influence it had upon the final technical project, and (2) 
according to the likelihood that the project would be implemented. 
Dollar values derived from the model are then used to determine 
cost/benefit by several comparisons. For example, when the model 
was used to evaluate requests for assistance from CEL, technology 
transfer and utilization program showed a return of $2.72 for each 
dollar spent. 

Other comparisons, meaningful to a particular organization 
are suggested. 

It is believed that this study demonstrates that it is 
possible to meaningfully quantify in dollars a significant portion 
of the benefits of technical information and/or technical recom- 
mendations that are often identified as intangible. 

This benefit evaluation decision model should be particu- 
larly useful in evaluating the benefits of technology transfer 
and utilization programs in other organizations. 
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